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Executive Summary 


he Southern Education Foundation’s Minority-Serving Institution (MSI) Consortium for 

Innovation and Change was instituted in 2011 to advance creative and promising initiatives 
that enhance institutional practice and student outcomes. In keeping with the SEF mission, these 
innovations specifically address educational barriers that disproportionately affect the degree 
seeking low-income and student of color populations operating under the purview of higher edu- 
cation. Our commitment to giving good ideas a fair test leads this work in hopes of identifying a 
set of best practices that may be cultivated within the education community as a comprehensive 
set of interventions to best serve those who inspire our work and need it most. 


With such a vision in mind, this project was devised to target the persistent problem that 
developmental education (DE) poses for many, as both an impediment to degree attainment as 
well as an issue that disproportionately impacts low-income students and students of color. SEF 
engaged five Minority-Serving Institutions along with the Technical Review and Support Team 
(TRST) to launch an eclectic set of DE innovations. These five campuses represented both two- 
year and four-year, and public and private colleges and universities across the southeastern and 
southwestern United States. Additionally, the campuses included Historically Black Colleges and 
Universities (HBCUs) and Hispanic Serving Institutions (HSIs) creating a window into a space 
that is often understudied but is home to a significant number of our target population. All 
involved eagerly engaged in this work to answer the pressing question of what can be done to 
mitigate the barriers to degree attainment for those who currently require DE. 


The activities of the MSI Consortium for Innovation and Change impacted nearly |,000 students 
who participated in the initiatives. Further, this experience of the pilot projects offered significant 
lessons learned and serious implications for the nearly 40,000 students currently attending the 
participating campuses, and unknown indirect benefits to countless future students. Armed with 
financial support and technical assistance, the campuses were empowered to implement a new 
DE strategy as well as a rigorous evaluation process that included experimental and quasi-ex- 
perimental research methods. The participating MSls utilized models that either accelerated or 
eliminated the semester-long DE course for first-year students, or focused on curricular redesign 
and faculty development. Promising models for change were observed in this first cohort of MSls 
that both enhanced the academic achievement (i.e. course completion and performance) and 
diminished the time to credit-bearing courses for the participants who tested into DE. 


This report includes a summary of the DE interventions, evaluation strategies, and program 
outcomes performed at the participating institutions. Additionally, it offers descriptions of the 
evidence-based models that have been effective for DE at the MSIs. The report goes on to high- 
light effective strategies utilized by the campuses in an effort to scale the initiatives and lessons 
to consider for scaling such models. Finally, it concludes with recommendations for advancing 
developmental reform efforts to improve teaching and learning while simultaneously addressing 
equity issues. 


Introduction 


BD ae somewhat groggy, wakes up extremely early from a night of tossing and turning in 
eager anticipation of her first day of college. Today, she will arrive on campus expecting to 
select her favorite classes and to learn to navigate her way around campus. However, when she 
arrives at the advising office, rather than discuss what she hopes to study and why, she is told that 
she must complete a placement test to determine which courses she can enroll in. This exam will 
determine whether Denise is officially dubbed “ready for college courses” despite already having 
been admitted. Thinking her biggest challenge that day would be finding the right building, Denise 
is startled into the reality that all too many students face. Low-income students and students of 
color, having beat a number of odds to make it to campus, often find yet another set of barriers 
slowing their progress toward college completion. 


Generally, students whose test scores or grades suggest that they require additional academic 
development before they can be successful in college-level courses are required to take remedial 
or developmental education (DE) courses. DE courses have generally been offered to improve 
skills and competencies required to help students become “college ready,” or ready to “succeed 
in entry-level, credit-bearing college courses without the need for remedial or developmental 
coursework” (Conley, 2012). Far too often students like Denise, not only are faced with the 
surprising factor that the same college that assessed them as “ready” enough to be accepted, 
have deemed them not “ready” for college courses. This is an unexpected conundrum for most 
and serves as a discouraging introduction into the college experience for far too many. In 2006, 
more than 50% of students at two-year institutions, and nearly 20% at four-year institutions, were 
enrolled in DE courses (Complete College America, 2012; Sparks & Mallus, 2013). 


As if that isn’t startling enough, the developmental level courses that students like Denise must 
complete to be admitted to the college level, credit-barring courses, often create a number of 
obstacles before they start. Fewer than half of these students actually pass these classes and 
successfully move on to credit-bearing courses that count towards a degree. Among those who 
do pass, far too many do not successfully complete the entry-level course that follows (Radford 
& Horn, 2012). DE courses cost the same as any other three-credit course but typically do not 
count toward a degree program. Thus, the problem of low DE and subsequent college-level 
course completion rates are inextricably linked to retention and degree completion, especially 
among low-income students and students of color who already have thin margins of error 
financially. Taking DE courses can mean that students are paying for courses that will not count 
towards graduation, which can explain why students enrolled in these courses often have fewer 
credits earned and lower retention and completion rates compared to their peers that are not 
required to complete DE courses (Complete College America, 2012). Further, DE courses are 
often instructed by faculty who are given the least support in way of salary, professional devel- 
opment, and on-campus resources such as office space. As a result there is often high turnover 
among DE faculty (Bickerstaff & Cormier, 2015; Boyer, Butner, & Smith, 2007). 


These challenges with DE as a barrier to college access and success are exacerbated for students 
who already face the greatest barriers, low-income students and students of color. For example, 
56% of African American students, 45% of Hispanic students, and 55% of Pell grant recipients 
require DE courses (Complete College America, 2016). It is important to note that this issue 

is persistent across institutional type; therefore, students of color and low-income students are 
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overrepresented in DE courses, not only at institutions that primarily serve these populations 
like community colleges and MSls, but also four-year, Predominately White Institutions (PWIs). 
Nearly 100% of two-year institutions and 80% of four-year institutions offer and enroll students 
in DE courses (Bettinger, Boatman, & Long, 2013; NCES, 2003; Sparks & Malkus, 2013; Pretlow, 

& Wathington, 2011). For four-year PWls, DE remains a contributor to persistent equity gaps in 
completion by race and income (Eberle-Sudre, Welch, & Nichols, 2015).The student level chal- 
lenges of accessing high quality instructional experiences, limited financial resources, and delays to 
completion are compounded for the institutions with limited resources and concentrated popu- 
lations of these students, MSIs and community colleges. These students who test into DE courses 
are largely concentrated at campuses like community colleges and MSls that often have admis- 
sions policies aimed at providing opportunity, but due to budget constraints, can be challenged by 
the need to simultaneously offer developmental and advanced credit-bearing courses (Bustillos, 
2012). These factors contribute to the context at MSIs where on average 70% of students require 
at least one DE course and less than 50% of students complete a bachelor’s degree within six- 
years (Bustillos, 2012; Li & Carrol, 2007; Sparks & Malkum, 2013). 


Therefore, reforming DE is one of higher education’s most critical equity imperatives across 
institutional type. Consequently, the current push in higher education to make college level, cred- 
it-bearing courses more accessible to all students, but especially students of color and low-income 
college students, is the single most significant action being taken to dismantle structural inequality 
in higher education. While there are several other issues that are critical to college access and 
success for low-income students and students of color, there lacks a similar momentum from 
philanthropic, state and federal policy, and campus communities to support experimentation, inno- 
vation, and reform. In the past decade, there has been a plethora of initiatives and studies focused 
on DE at community colleges. These efforts were essential in informing the field that: 


|. Students are more likely to complete college if they are engaged in work that counts toward a 
degree or credential in their academic or career area of interest. 


2. Students at all levels of preparation need support with college-level courses and co-requisite 
models can provide a variety of modified types of support to students. 


3. Gateway courses that include mandatory support benefit students more than the traditional 
course models. 


4. Improved partnerships between college-preparatory and postsecondary programs benefit 
students entering college and limit the need for extra support. 


(Core Principtes, 2015) 


Missing from this work on DE reform is an emphasis on four-year colleges and universities 

and two-year and four-year MSls. DE programs are too expensive for students and too costly 
for institutions to know so little about which program models actually work in these critical 
contexts. As a result, in 201 1, the Southern Education Foundation (SEF) created the Minority- 
Serving Institution Consortium for Innovation and Change to support creative and promising 
initiatives that improve institutional practices and student outcomes. This project coalesced and 
supported a group of MSls intensely focused on the development of solutions to a common 
and persistent problem that impedes degree completion and DE.The participating MSls utilized 
models that either accelerated or eliminated the semester-long, DE courses for first-year stu- 
dents, or focused on curricular redesign and faculty development. Found in this first cohort of 
MSlIs were promising models that increased the academic achievement and decreased the time to 
credit-bearing courses for the participants who tested into DE. 


Evidence-Based Models for DE Success at MSIs 
Overview 


Ithough slightly different, each campus participating in the MS! Consortium for Innovation 

and Change executed a plan for supporting the academic success of first-year students 
whose level of academic preparation suggests they would benefit from additional academic 
support in order to be successful in college-level courses. More specifically, each model employs 
a strategy aimed at increasing the success of these students in their first-year college-level math 
and/or English courses by providing them with condensed DE courses through summer bridge 
programs or academic support via structured labs as they enroll in college-level courses. These 
strategies represent evidence-based approaches to course acceleration, with research suggesting 
that these approaches significantly decrease the time spent in DE courses without threatening 
success in credit-bearing courses, consequently reducing the time it takes for these students to 
complete a college degree (Jaggars, Edgecombe, & Stacey, 2014). 


In the Fall 2013 semester, the campuses collaborated with SEF and the technical review and 
support team to launch various DE innovations. Once concluded, they continued tracking par- 
ticipant outcomes through 2014. Each campus was provided with an initial grant of $25,000 to 
support the launch of the innovation and its subsequent study. Upon successful implementation 
of the innovation and cooperative participation in the consortium they were provided with a 
second $25,000 grant. The campuses used summer bridge programs, fast-track course models, 
extended instructional time, cross-course curricula integration, supplemental instruction, and 
mainstreaming as approaches to increase the success of students testing into DE courses. The 
program activities are described in greater detail in Figure | below. 


“The findings from this group of 
pilot studies and the data from DE 
reform across the nation suggest that 
students who were previously labeled 
as “underprepared” or “not college 
ready” can be successful in college 
level, credit-bearing courses, without 
traditional DE courses” 
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Figure |. DEVELOPMENTAL EDUCATION INNOVATIONS 
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Claflin University 
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Texas Southmost College 


TRADITION * INNOVATION « OPPORTUNITY 


Texas Southmost College 
(TSC) 


University of the Incarnate Word 


(UIW) 


Intensive Math Program (IMP) (created in 2001) allows participating 

students who require one or two DE math courses the opportunity 

to finish their DE course requirements prior to their first semester 

by completing the five-week summer bridge program. Students 

receive instruction and intensive, directed tutoring. With support 

from SEF, CSUSB was able to increase enrollment and evaluate the 
program for improvement and development. 


Claflin’s “Extended Course” model (created in 2010) which 
combined former DE courses with credit-bearing courses in 
mathematics and English, are taught five days per week and include 
tutorial support and computer lab time for mathematics.“Extended” 
English instructors collaborate with the Writing Center to 
strengthen instruction and address skill deficiencies in composition. 
With support from SEF Claflin was able to design and incorporate 
performance based academic assessment strategies into the courses, 
and formalized academic support strategies that involved the writing 

center and associated tutors. 


With support from SEF MSU created a Developmental Humanities 
Initiative which includes hands-on interdisciplinary projects across 
the disciplines of Reading, English and History. Faculty members, 
including non-tenure track, engaged in course re-design to create the 
interdisciplinary syllabi across DE and non-DE courses. The syllabi 
were also designed to increase student engagement and academic 
identity development by incorporating culturally relevant content. 


With support from SEF TSC developed an accelerated 

developmental math course which combines Introductory and 

Intermediate Algebra courses into two consecutive 8-week sessions, 

rather than the traditional format of two consecutive |6-week 

sessions. Students were provided with a lab hour attached to each 

course, where they received support from the instructors and tutors 
and completed online assessments. 


With support from SEF UIW created a Co-Requisite model of 

college-level College Algebra course with a mandatory lab for 

students requiring DE. The additional hour gave the instructor the 

opportunity to remediate based on specific deficiencies indicated 

by the placement exam while still meeting college-level learning 

outcomes. Participating students were described as “bubble students” 
who just missed the college-level math cutoff score. 
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Outcomes 


SEF and the technical review and 
support team worked closely with the 
campuses to examine the effectiveness 
of the innovations, with two campuses 
executing random assignment and 
three utilizing comparative analyses. 
The findings demonstrate that on 
average the students who would 
otherwise be required to complete 

a traditional DE course sequence, 
instead participated in the innovations, “In Some Cases, 
and had either similar or better course 


outcomes than similar students who the students 

require DE, but did not participate participating in the 
in the innovations. In some cases, the Z ‘ 

students participating in the innova- interventions had 
tions had similar to better outcomes similar to better 

than even those students who did 

not require DE. For example, at the outcomes than even 
University of the Incarnate Word, stu- those students who 
dents who would have otherwise been ‘ ; 

placed in a traditional DE course but did not require DE.” 


were enrolled in a college-level math 
course with an additional math lab, had 
similar final course grades and slightly 
lower final test scores than students 
who did not test into DE courses. At 
Morgan State University, students who 
participated in the integrated courses 
had higher attendance, course comple- 
tion rates, grades, and final test scores 
than similar students who did not par- 
ticipate in the integrated course model. 
See Figure 2 for outcomes summary. 
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Figure 2. PRoGRAM OuTCOMES 


Innovation 


98.7% of the students were able to successfully test out of 


(at the required DE course after completing the summer math 
CALIFORNIA STATE UNIVERSITY 
SAN BERNARDINO program. After the program, the academic performance (grades) of 
the participating students in college-level math was on par with their 
California State University-San “college ready” peers. The findings suggest that the summer bridge 
Bernardino program was an adequate replacement for the traditional DE math 
course. 


Students in the program enrolled in the extended English 101 
course had a lower passing rate than the control group of similar 
students participating in the traditional English 101 course (88% 
vs. 99.3%). However, for those students who did pass and go on to 
the next course, English 102, students in the program had higher 
Claflin University course completion rates (79% vs. 67%) and grades than the students 
who participated in the control group during the previous semester. 


Students in the program group had higher pre and post- 
test scores and higher attendance than the control group. 
Attendance was also shown to be correlated with grades. The 
ex program participants also self-reported higher levels of satisfaction 
MORGAN with the courses than the control group students participating in the 
soapaearairs traditional model. The findings suggest, the integrated course model 
has positive impacts on student engagement, which could help explain 
some differences in test scores. However, because the control group 
had higher pre-test scores, the understanding of the impact of the 
program on post-test scores is limited. 


a 


Morgan State University 


The students in the accelerated math program had higher 


retention, course completion rates, and grades, than the 

students in the traditional course model. However, the college- 

Texas Southmost College level math course completion rates are slightly lower for the students 

oer as a who participated in the accelerated model. The findings suggest, 
Texas Southmost College higher short term benefits of the program. 


Students who would have otherwise been placed in a traditional 

DE course but were enrolled in a traditional college-level math 

course with an additional math lab had similar final course 

grades and slightly lower final test scores than students who did 

not test into DE courses. Findings suggest the college-level math 

course with additional support was an adequate substitute for the 
traditional DE math course. 


University of the Incarnate Word 


Model 1: Summer Bridge Program 


he first model, a summer bridge program at California State University San Bernardino 

(CSUSB), is a supplemental program, aimed to enhance students’ academic content knowl- 
edge and the other cognitive and non-cognitive skills needed for academic success like study 
skills, time management, and social adjustment (Rutschow & Schneider, 201 1). CSUSB is a public 
four-year Hispanic Serving Institution (HSI) with a population of 16,743 undergraduate students. 
Over 70% of CSUSB students are Pell grant eligible and first-generation college students, and 
over half are Latino or Hispanic. On an annual basis, 69% of first-time, full-time freshmen at 
CSUSB require DE courses. For over a decade, CSUSB has offered students the opportunity to 
complete their DE course requirements prior to their first semester by engaging in the summer 
bridge program and successfully testing out of the DE course. 


The assessment of program effectiveness revealed that the participating students had similar 
first-year GPAs and attempted and completed more credits in the first year than demographi- 
cally similar peers who were exempt from DE math (See Figures 3-5 below).The participating 
students also took less time to complete their general education math courses than their peers 
who require DE math and those who do not. Most recently, over 90% of the 755 students who 
participated in the summer 2015 program successfully tested out of DE math after the comple- 
tion of the bridge program. 


ON 


CALIFORNIA STATE UNIVERSITY 


Figure 3. CSUSC ParticipANT DEMOGRAPHICS SAN BERNARDINO 
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Figure 5. CSUSB ParticipANT 
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Model 2: Co-Requisite Courses 


he second model was a co-requisite model used by the University of the Incarnate Word 
(UIW).This model allowed DE students who were just a few points short of testing out 
of DE courses, also known as bubble students, (see Figure 6) to enroll in college-level courses 
while simultaneously receiving targeted academic support. UIW is a private four-year HSI with 
a population of 6,496 undergraduate students, over half of whom identify as Hispanic or Latino. 
Approximately 47% of UIW students are Pell grant recipients. The average GPA for UIW students 
is 3.47, and the average SAT score is 975. 


Figure 6. UIW Bussce STUDENT PROFILE 


ACCUPLACER 


College Level 


Students who meet any (1) of 
the scores 


Quick Start 


490-519 
“The Bubble Student” 


0318/0319 (as directed 
by requirements for 


individual major) < 490 <19 <57 


Students who do not meet any 
(1) of the scores 


The targeted academic support for these bubble students was provided during a required math 
tutoring lab, which was compulsory for one hour per week. During the math lab the students 
were expected to complete ALEKS modules and demonstrate mastery of the math course 
objectives using the ALEKS exit exam. The additional hour of tutoring gave the instructor oppor- 
tunities to remediate based on specific deficiencies outlined by the Accuplacer results. This was 
accomplished while still meeting college-level learning outcomes. Non-cognitive issues like self-ef- 
ficacy, motivation, and study and note-taking skills were also addressed during the tutoring ses- 
sions, specifically through the use of guided math challenges. Essentially, students who would have 
otherwise been placed in a traditional DE course but were enrolled in a traditional college-level 
math course with an additional math lab (program group) had similar final course grades and 
slightly lower final test scores than students who did not test into DE courses (control group) 
(see Figure 7 below). 
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Figure 7 UIW Co-ReEaquisitE MopEL Outcomes 
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Model 3: Interdisciplinary Curricula 


he third model is Morgan State University’s (MSU) collaborative approach to DE in their 

Humanities department. MSU is a public four-year Historically Black College and University 
(HBCU) with a population of 6,302 undergraduate students, over 80% of whom identify as Black 
or African American. Approximately 60% of first-year students at MSU require DE courses. The 
MSU Humanities faculty collaborated to integrate the curricula of their developmental level Reading 
and English, and college-level History courses to enhance student learning and outcomes across 
the three courses. Faculty, that included tenured and non-tenure track faculty collaborated on 
curricula redesign with an emphasis on enhancing student engagement through the use of culturally 
relevant texts and topics. Students were placed into World History and Developmental English and 
Developmental Reading courses based on their Accuplacer Placement Test results. The team used 
a non-experimental design in which they compared the outcomes of the students assigned to the 
revised curriculum courses (the program group) to the group of students that received the tradi- 
tional curriculum (the control group). Each of the six courses (three program and three control 
courses) were limited to 25 students each. The findings demonstrate that overall the students who 
participated in the integrated curricula courses had higher course completion rates, better atten- 
dance, and higher grades and post-test scores than their non-participating peers who were enrolled 
in traditional DE courses. These findings are provided in greater detail in the Figures 8-11 below. 


a) 


Figure 8. MSU FiNnaAL Course GRADES MORGAN 
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Figure 9. MSU Course CompPLeETION 
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Figure 10. MSU Pre/Post Test Scores 
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Figure Il. MSU NumsBer or ABSENCES 
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The findings suggest a positive effect or an increase in student outcomes associated with partic- 
ipating in courses with an integrated curriculum. An integrated curriculum has the potential to 
positively impact learning gains because of its ability to reinforce concepts and increase students’ 
opportunities to apply academic concepts. This transferability of knowledge from one course to 
another was observed by faculty who reported that students started to acknowledge the overlap 
in the curricula and would ask “are you working with the history teacher because | swear we just 
talked about that in history.’ 


One could hypothesize that students may see a greater value in investing time in the courses and 
associated work because they view their efforts as “double counting” or a multiplied benefit as a 
result of the overlapping curricula. One faculty member even hypothesized that the overlapping 
curricula helped boost students’ confidence when she stated “seeing how the classes overlap and 
giving them a sense of confidence to speak out when ordinarily they may not have spoken out for 
fear of being wrong or not understanding the subject matter—I think this cross-over or hybrid 
course has boosted their self-esteem in the classroom.” Increases in confidence or engagement 
observed by the faculty, may also have contributed to the great reduction in the persistent 
problem of unexcused absences seen in the traditional courses. 
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Model 4: Extended Courses 


laflin University is a private four-year HBCU with a population of 1,800 students. 

Approximately 60% of Claflin’s students are first-generation college students, over 90% are 
African American or Black, and nearly 90% are Pell grant recipients. In 2010, when faced with 
the state imposed mandate to eradicate developmental English courses, Claflin University began 
placing students whose test scores suggest they would benefit from additional academic support 
into an extended course model. Institutional assessment data indicated that most of the students 
who would place in the former DE course scored particularly low in reading. Subsequently, 
students were placed into a college-level English 101 course that met 3 days per week, approxi- 
mately 45 contact hours, and extended the course by two days integrating more reading instruc- 
tion and increasing the contact hours in the course to 75 hours. 


In the Fall semester of 2014, approximately 116 students were placed into 8 sections of the 
extended English 101 courses. The students were randomly placed in either a program group that 
included writing center assistance, or a control group that followed the traditional model but did 
not include the writing center emphasis. 


Writing Center 


Frequent visits to the writing center to monitor the writing process was a mandated intervention 
for the program group. At the writing center, the students received individualized assistance from 
a group of peer tutors led by a faculty member from the English department. The control group 
was not mandated to go to the writing center. However, later it was discovered that instructors 
in the control group offered grade incentives for students to attend. As a result, 90% of the 
program group and 71% of the control group registered attendance of at least nine visits to the 
writing center during the fall. Disruptions in the leadership of the writing center and the discon- 
tinued instructional course and ongoing meetings, made it difficult to adequately monitor the 
groups in the study. There was also a breach in the method of adequately preparing peer consul- 
tants for supplemental instruction specific to the needs of basic writers in the study. 


The outcomes suggest that the students who participated in the initiative were successful as 
indicated by their course passing rates that were above 80%; however, there is limited evidence 
that the program students were more successful than the control group students who had higher 
course passing rates that averaged above 90%. The program group has higher grades than the 
control group in the following course (English 102), thus the impact of the program on student 
outcomes, may be delayed. The outcomes, including the extended course pass rates and perfor- 
mance in the following course, English 102, are provided in Figures |2-14. Claflin University is 

in the process of conducting an analysis of the impact of the transition from the traditional DE 
model to the extended course model. The current assessment is only reflective of the impact of 
the writing center and tutoring strategy on the extended course model. 
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FIGURE 12. CU Finat Grape ANALysis-ENG 101 
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Figure 13. CU Course Grapes-ENG 102 
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Figure 14. CU Course Grapes-ENG 102 
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Model 5: Accelerated Math 


he Fast Track program consisted of an accelerated learning model offered to students 

enrolled in developmental math courses at Texas Southmost College (TSC) starting in Fall 
2013.TSC is a public two-year HSI with a population of about 4,000 students, with over 90% of 
its students identifying as either Latino or Hispanic. Acceleration refers to the reorganization 
of instruction and curricula into 8-week courses in order to expedite the completion of DE 
coursework. TSC students had the opportunity to complete two levels of DE coursework in 
one |6-week semester. That is, the accelerated courses consisted of two 8-week math courses 
that run consecutively in a single semester. Two developmental mathematic courses were 
included in the accelerated 8-week learning model: Introductory Algebra and Intermediate 
Algebra. The Introductory Algebra course was conducted during the first 8 weeks and the 
Intermediate Algebra course was conducted during the second 8 weeks. This sequence allowed 
students to complete the developmental math sequence in one semester. 


Participants in this program were TSC students who self-selected to enroll in either two “tra- 
ditional” |6-week developmental math courses offered in two consecutive semesters or two 
consecutive 8-week “accelerated” developmental math courses within the same semester. By 
tracking developmental math students through these two different sequence models, TSC was 
able to determine how developmental math students performed in college-level math courses. 
The following graphs summarize the grades and passing rates for both the accelerated and tra- 
ditional programs. Although, there are limitations because approximately 600 participants from 
the Fall 2013 cohort self-selected into the course options, the findings demonstrate that those in 
the accelerated courses had better grades, course completion rates, and retention (consecutive 
course enrollment) than those enrolled in the traditional course model (see figures 15-20 below). 


Figure 15. TSC Course Grapes SUMMARY Figure 16. TSC Passinc AND FAILING GRADE RATES 
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Figure 17. TSC Course CompLetion Rates Fatt 2013 Figure 18. TSC Course CompLetion Rates-SpriNG 2014 
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Research and Evaluation 


he aim of the MSI Consortium for Innovation and Change was not only to support innova- 

tive approaches to DE, but to also ensure that defensible evidence about the effectiveness of 
these innovations was produced. The members of the technical review and support team guided 
the design of a research and evaluation study that would discover whether various innovations 
were effective, and if so distinguish in what ways it was successful. A description of the technical 
review and support team members is found in Appendix A. Along with SEF staff, the technical 
review and support team made trips to each campus to accomplish a key set of tasks: 


|. Convene the key institutional leaders and faculty to discuss the progress of the program. 
2. Address any challenges related to the evaluation study. 


3. Discuss preliminary findings and next steps for continuing or expanding the program at the 
campus. 


A summary of the research and evaluation strategy for each campus is provided in Figure 21 
below: 


Figure 21. EVALUATION STRATEGY 


Campus Evaluation Strategy 


* Accomparative analysis of 12 years of data on Intensive Math Program 
(IMP) participants and non-participants. IMP is a math intensive program 
taking place for five weeks in the summer before students’ fall semester. 


* Could be classified as acceleration as the goal is to provide enough in- 
California State University-San struction in five weeks so that students no longer need DE by the start 
Bernardino of the fall semester. 


Summer Bridge Program * The available data includes grades in general education math course- 
work, retention rates, total units completed, total general education 
units completed, Ist year GPA, graduation rates for nearly 400 IMP 
participants. Collected qualitative data on the implementation of the 
program and participant experiences. 


¢ A randomized study of the impact of the Extended English intervention 
on student outcomes including post-test outcomes, portfolio evalua- 
tions, attrition, and course grades. 


Claflin University * Extended English participants for fall 2013 were enrolled in college-level 
English three days a week and participated in two additional days of 
instruction to remediate their skills (acceleration) so they did not need 
to enroll in a complete DE course. 


Extended Courses 


¢ The control group were spring 2014 non-developmental English course 
participants. Approximately | 16 students were included in the study. 
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Campus Evaluation Strategy 


¢ A comparative analysis of the impact of an integrated curricula 
approach to DE courses on outcomes like, DE course completion rates 

: : and grades, retention, and general education course enrollment and 

Morgan State University cotnpletion 


Interdisciplinary Curricula Rn a : : ; 
P y The initiative includes an integrated curricula approach to DE courses in 


reading and English and the general education history course. Approxi- 
mately |22 students were included in the study. 


* A comparative analysis of the impact of their “fast-track” developmental 
math course which combines Introductory and Intermediate Algebra 
Texas Southmost College courses into two 8-week sessions on student outcomes (acceleration). 


Accelerated Math * TSC compared developmental and gateway course success of fast- 
track students with students who took the traditional |6-week format. 
Approximately 400 students were included in the study. 


¢ A randomized study of the impact of the co-requisite course approach 
on student outcomes including course grades and completion, retention, 
and pre/post test scores (ACCUPLACER). 


University of the Incarnate Word ¢ Participants for fall 2013/spring 2014 were enrolled in college-level 


Co-Requisite Model math, but were provided with a one hour math lab to address their 
remediation needs (acceleration) so they did not need to enroll ina 
semester long, traditional, DE course. Approximately 25 students were 
included in the study. 


The technical review and support team was key to helping the campuses develop and imple- 
ment a rigorous research design and evaluation study. At UIW and Claflin University, this meant 
randomly assigning students to intervention and treatment groups in order to account for 
factors other than the intervention that could impact observed student outcomes. The diffi- 
culty of executing experiments on college campuses is well documented and is often caused by 
limited capacity, resources, and logistical challenges. Further, the inability to control for student 
choices in college courses, and program enrollment; the non-conformity of instructional delivery; 
curricula; course materials; and the coordination of “loosely-coupled” administrative and aca- 
demic units pose additional challenges. Consequently, SEF and the technical review and support 
team members worked together to provide each campus with support to ensure fidelity to the 
research and evaluation study. 
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Lessons for Scaling DE Initiatives 


he campuses in SEF’s MSI Consortium for Innovation and Change, along with other higher 

education leaders, policy makers, and partners across the nation are exploring how to 
improve academic success and degree completion through the improvement of DE. Therefore, it 
is critical to highlight a few lessons learned for scaling strategies similar to those piloted by the 
campuses in the MSI Consortium for Innovation and Change. 


1. One size does not fit all, but no need to dress the same 


The main benefit of scaling an evidence-based strategy is being able to impact outcomes for 
significant groups of students, the institution, and higher education as a whole. However, the 
primary challenges of scaling are the limitations in being able to meet individual student needs. 
For example, the findings from UIW suggest that students who fall just below the recommended 
margin for inclusion into credit-bearing college courses are successful in the co-requisite model 
that places them directly into a college-level course. In such cases, a one hour math lab provides 
sufficient support. These findings support other studies (Complete College America, 2016; Jaggars 
et. al., 2014) that suggest co-requisite models can be an effective replacement for traditional DE 
courses. However, it is important to emphasize that in this case it was for a particular type of 
student, the “bubble student” who tested somewhat close to the cutoff score required to enroll 
in college level math courses. Our evidence also suggests that students whose test scores suggest 
that they would benefit from multiple DE courses, are likely not best suited to skip over the 
courses entirely. Instead they could benefit from accelerated models like those used by TSC. In 
both cases, the students were able to complete their DE requirements with greater success, in 
less time, and without threatening academic performance later in their time at the university. 


Thus, these examples demonstrate the importance of offering multiple DE options for the 
multiplicity of student needs that may exist on a single campus. Although it could take concerted 
effort and resources for campuses to transition to offering one or more of these options, they 
will see a return on their investment in the form of increased retention and completion rates. 
However, the effectiveness of these strategies depend on continuous improvement of placement 
tests, and methods for assessing students’ skills, thus SEF provided support to campuses like UIW 
to continue experimenting with different placement exams. The aim is to enhance correct place- 
ment, which will inform the refinement of academic support strategies, course performance, and 
student success beyond first-year courses. 


2. Promoting intercampus collaboration is critical 


DE should not be singularly relegated to academic support offices or departments. In fact, 

DE impacts various divisions of academic and student affairs. Thus, these different parts of the 
campus must be engaged to ensure new DE strategies are successfully implemented and evalu- 
ated. Getting buy-in from senior academic and administrative leaders is key to ensure organiza- 
tional support and necessary to coordinate multiple academic and student services departments. 
This buy-in is critical in helping to navigate unexpected hurdles and to keep the project moving 
forward in the face of competing demands. MSU’s integrated curricula design was an example of 
a DE strategy that was dependent on the collaboration of faculty, campus leaders, and academic 
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and student services departments. Beyond engaging different types of faculty across departments, 
MSU noted the importance of working with the registrar to evaluate program impact using 
random assignment. 


3. How it works is as important as what works 


It is important to create innovations in DE; however in doing so, it is helpful to draw on existing 
theory and research about student learning, engagement, and outcomes to develop the innova- 
tions. It is helpful to create campus level knowledge on existing DE challenges, and outcomes 
from pilot programs, thus context relevant adjustments can be made before bringing the initia- 
tives to scale. Therefore, it is critical to think through a plan for assessing program impact prior 
to implementation because the requirements of rigorous evaluation research can inform program 
design. Program assessment is critical at MSIs because existing research on DE is not always 
inclusive of MSls or specific to the experiences of Black or Latino students. Lastly, use of qualita- 
tive and quantitative strategies to study program impact is critical to telling a robust story about 
if, and how initiatives are working. Qualitative methods, such as interviews and focus groups, can 
explicate quantitative phenomena and provide insight into issues ranging from program imple- 
mentation and fidelity to program design, both of which are critical to understanding the impact 
of an institution. The qualitative methods can be especially helpful in understanding how these 
particular initiatives work at MSIs and for students of color. 


“These examples demonstrate the 
importance of offering multiple 
DE options for the multiplicity of 
student needs that may exist on a 
single campus” 
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Conclusions 


ome states and institutions have opted to eliminate or reduce DE course offerings. This has 

led to rapid changes in approaches to DE practices. Much of the programmatic work aimed 
at improving DE has taken place in community colleges and has produced mixed results. There 
are many program designs, theories, and practices currently in play. Some involve accelerated 
courses, computer assisted instruction, prescribed course sequences, tutoring supplements, the 
use of math labs, writing groups, or the option to test out of a developmental course at any time 
during a semester. Campus-based efforts to assess the effectiveness of their DE programs vary 
tremendously in concept, rigor, reliability, and relevance to the larger higher education commu- 
nity. SEF’s MSI Consortium for Innovation and Change utilized rigorous methods of evaluation 
and assessment that included random assignment and matched comparison groups, in order to 
produce defensible evidence about what works for supporting the academic success of underpre- 
pared students in their first-year college courses. Based on this work, final thoughts pertinent to 
continuing higher education’s efforts to address DE are offered. 


Developmental Education reform should include an emphasis on teaching and 
learning. 


Although structural reforms, such as transitioning to co-requisite or other accelerated models 
are critical, instructional strategies and academic support remain critical issues that must be 
addressed in these shifts. Stakeholders interested in supporting DE reforms that emphasize 
teaching and learning should consider the following: 


a. What are the best strategies for providing supplemental academic support to students who 
would have previously been required to complete a traditional DE course sequence? 


b. How can campuses engage faculty who have experience instructing DE courses, to enhance 
the teaching strategies used in the new course models? 


c. How can all courses, but especially courses for first-year students incorporate culturally 
relevant strategies in order to facilitate greater student engagement? 


d. How can the integration of content across courses enhance student learning and engagement, 
especially in first-year courses for students that may require additional academic support? 


“,..initiatives like the push for the “right” math 
and guided pathways, could easily become 
vehicles for tracking low-income students and 
students of color into what are perceived as 
“less rigorous” pathways.” 
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Developmental Education reform efforts must center on equity issues. 


We should not implement structural changes (i.e. transitioning from traditional to co-requisite 
models) without explicitly addressing higher education’s persistent challenges assessing student’s 
academic abilities, especially as it pertains to low-income students and students of color. As the 
issue of disproportionately labeling low-income students and students of color as “remedial,” 

is being mitigated by DE reform, higher education must continue to wrestle with why these 
students were disproportionally labeled remedial in the first place, and how low expectations of 
these populations permeate other aspects of higher education such as advising and discipline. The 
findings from this group of pilot studies and the data from DE reform across the nation suggest 
that students who were previously labeled as “underprepared” or “not college ready” can be suc- 
cessful in college level, credit-bearing courses, without traditional DE courses, which challenges 
higher education’s often deficit view of these students. Therefore, the higher education commu- 
nity must explicitly acknowledge that we were wrong to assume that many low-income students 
and students of color could not be successful in college level, credit-bearing courses. Otherwise, 
structures may change, but low expectations and inequities will persist. For example, initiatives 
like the push for the “right” math and guided pathways, could easily become vehicles for tracking 
low-income students and students of color into what are perceived as “less rigorous” pathways. 
Therefore, stakeholders interested in DE reform that centers equity on issues should consider: 


a. Have we examined whether these DE reform strategies are improving or challenging equity 
as it pertains to the academic outcomes of low-income students and students of color? Are 
equity gaps closing or widening? 


b. Have we incorporated strategies to educate faculty, advisors, and other campus leaders on 
equity issues, especially as it pertains to our perceptions of the academic ability of students of 
color and low-income students? 


c. Are there any unintentional consequences of these DE reform efforts for low-income students 
and students of color on campus? How are these populations performing once they exit their 
first-year courses? Are they enrolling in STEM courses at similar rates? 


d. How are we engaging, learning from, and enhancing DE at the institutions that are most likely 
to enroll low-income students and students of color, i.e. Minority-Serving Institutions? 


These questions have been provided to assist key stakeholders, which include funders, policy 
makers, campus leaders, faculty members, and key intermediary organizations as they continue 
the critical work of reforming DE and potentially enhancing college access and outcomes for 
low-income students and students of color. The continued improvement of DE requires coordi- 
nation and support from the diverse array of higher education advocates. Although DE reform 
requires a substantial investment in time, effort, and resources, the result could be the most 
significant contemporary effort to dismantle structural inequality in higher education. 
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Appendix A 


Technical Review and Support Team 


Dr. Stephen Porter 


Dr. Stephen R. Porter serves as Professor of Higher Education in the Department of Leadership, 
Policy and Adult & Higher Education at North Carolina State University, where he also serves 

as coordinator for the graduate research methods sequence for the College of Education. He 
teaches courses in educational statistics, causal inference with observational data, and survey 
research methods. He received his Ph.D. in political science from the University of Rochester, 
with a concentration in econometrics. Prior to his faculty positions at North Carolina State 

and lowa State University, he spent nine years in higher education administration in the field of 
institutional research, working first at the University of Maryland, College Park, and most recently 
as Director of Institutional Research at Wesleyan University in Connecticut. 


Dr. Josh Pretlow 


Dr. Josh Pretlow earned a PhD in Higher Education with a minor in Research, Statistics, and 
Evaluation from the University of Virginia in 2011. As a graduate student and postdoctoral 
researcher, Josh worked for the National Center for Postsecondary Research, an IES funded 
center focused on rigorously measuring the effectiveness of programs designed to help students 
make the transition to college and master the basic skills needed to advance to a degree. His dis- 
sertation was an experimental evaluation of a Summer Bridge Program designed to help recent 
high school graduates successfully transition to community colleges. His current research con- 
tinues to include the transition to higher education and includes a focus on dual enrollment and 


education policy. He is currently the Senior Research Analyst at the University of North Carolina 
at Chapel Hill. 


31 


MSI Consortium for Innovation and Change 


SOUTHERN EDUCATION FOUNDATION Developmental Education Report 


Dr. Leticia Tomas Bustillos 


Leticia Tomas Bustillos, Ph.D. is Associate Director of the Education and Children’s Policy Project 
at the National Council of La Raza and has over I5 years’ experience working across the K-20 
education pipeline. She served as Co-Director of the Policy Research on Preparation, Access 
and Remedial Education (PRePARE) Project and Associate Director of the Los Angeles County 
Education Foundation. She has been a consultant to several national based projects focused on 
developmental education, including the Getting Past Go Project and the Lumina-MSI Models 

of Success Program. Bustillos is the author of articles, book chapters and blog entries and is in 
the process of authoring two forthcoming books on critical action research and developmental 
education. She is a mentor with the College Bound Today program within the Montebello Unified 
School District and is a board member of Girls on the Run of Los Angeles County. She is a 
graduate of Columbia University (A.B., 1994), Whittier College (M.A., 2000) and the University of 
Southern California (Ph.D., 2007). 


Dr. Paul D. Umbach 


Dr. Paul D. Umbach serves as Professor of Higher Education in the Department of Leadership, Policy, 
and Human Development at North Carolina State University. He teaches classes on finance and 
higher education, organizational theory and higher education, policy analysis, college faculty, survey 
methods, and multi-level modeling. Prior to joining the faculty at NC State, he spent four years as an 
Assistant Professor of Higher Education at the University of lowa. Before joining the faculty at lowa, 
he served as a research associate with the Indiana University Center for Postsecondary Research. At 
the Center, he was a research team member working on the National Survey of Student Engagement 
(NSSE) and a project manager of the Faculty Survey of Student Engagement (FSSE). His professional 
background includes several years working in institutional research at Tidewater Community College, 
Old Dominion University, and the University of Maryland. He earned his Ph.D. in Higher Education 
from the University of Maryland, College Park. 
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